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Conclusion 


1  Introduction 


The  emergence  of  complex  dynamical  behaviors  from  simple  local  interaction 
rules  between  pairs  of  agents  in  a  group  is  a  widespread  phenomenon  over  a 
range  of  application  domains,  and  results  in  coherent  behavior  called  a  swarm. 
Many  striking  examples  can  be  found  in  biological  systems,  from  the  microscopic 
(ex.,  aggregates  of  bacterial  cells  or  the  collective  motion  of  skin  cells  in  wound 
healing)  [4,  32,  17]  to  large-scale  aggregates  of  fish,  birds,  and  even  humans 
[40,  14,  18].  These  systems  are  particularly  interesting  to  the  robotics  com¬ 
munity  because  they  allow  simple  individual  agents  to  achieve  complex  tasks  in 
ways  that  are  scalable,  extensible,  and  robust  to  failures  of  individual  agents.  In 
addition,  these  aggregate  behaviors  are  able  to  form  and  persist  in  spite  of  com¬ 
plicating  factors  such  as  communication  delay  and  restrictions  on  the  number  of 
neighbors  each  agent  is  able  to  interact  with,  heterogeneity  in  agent  dynamics, 
and  environmental  noise.  These  factors  define  how  swarm  behaviors  are  effected 
by  changes  due  to  internal  and  external  actors,  allow  the  swarm  to  change  its 
patterns  and  function  [36,  35,  37,  15],  as  well  as  break  apart  and  reform  [29]. 

A  number  of  studies  show  that  even  with  simple  interaction  protocols,  swarms 
of  agents  are  able  to  converge  to  organized,  coherent  behaviors.  Existing  litera¬ 
ture  on  the  subject  provides  a  wide  selection  of  both  agent-based  [14,  18,  43,  40] 
and  continuum  models  [10,  39,  32].  One  of  the  earliest  agent-based  models  of 
swarming  is  Reynolds’s  boids  [33],  which  simulates  the  motion  of  a  group  of  flock¬ 
ing  birds.  The  boids  follow  three  simple  rules:  collision  avoidance,  alignment 
with  neighbors,  and  attraction  to  neighbors.  Since  the  publication  of  Reynolds’s 
paper,  many  models  based  on  “zones”  of  attraction,  repulsion,  and/or  align¬ 
ment  have  been  used  as  a  means  of  realistically  modeling  swarming  behaviors 
[26,  38,  44],  Systematic  numerical  studies  of  discrete  flocking  based  on  alignment 
with  nearest  neighbors  we  carried  out  by  Vicsek  et  al.  [42].  Stochastic  interac¬ 
tions  between  agents  are  modeled  in  [31].  In  recent  years,  improved  computer 
vision  algorithms  have  allowed  researchers  to  record  and  analyze  the  motions  of 
individual  agents  in  biological  flocks,  and  formulating  more  accurate,  empirical 
models  for  collective  motion  strategies  of  flocking  species  including  birds  and 
fish  [2,  16,  5], 

Despite  the  multitude  of  available  models,  how  group  motion  properties 
emerge  from  individual  agent  behaviors  is  still  an  active  area  of  research.  For 
example,  [45]  presents  a  simulation-based  analysis  of  the  different  kinds  of  mo¬ 
tion  in  a  fish-schooling  model;  the  authors  map  phase  transitions  between  dif¬ 
ferent  aggregate  behaviors  as  a  function  of  group  size  and  maximum  number 
of  neighbors  that  influence  the  motion  of  each  “fish”.  In  [18],  the  authors  use 
simulation  to  study  transitions  in  aggregate  motions  of  prey  in  response  to  a 
predator  attack. 

Interaction  delay  is  a  ubiquitous  problem  in  both  naturally-occurring  and 
artificial  systems,  including  blood  cell  production  and  coordinated  flight  of  bats 
[22,  3,  28,  12,  11].  Communication  delay  can  cause  emergence  of  new  collective 
motion  patterns  and  lead  to  noise-induced  switching  between  bistable  patterns 
[23,  24,  20];  this,  in  turn,  can  lead  to  instability  in  robotic  swarming  systems 
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[44,  21].  Understanding  the  effects  of  delay  and  latency  is  key  to  understanding 
many  swarm  behaviors  and  functions  in  natural,  as  well  as  engineered,  systems 
[37].  As  another  controlling  factor  of  swarm  function,  network  communication 
topology  maybe  used  to  improve  and  generalize  swarm  function.  For  example 
in  [15],  it  was  shown  how  to  take  known  basic  swarm  modes,  such  as  flocking, 
milling  and  rotating  states,  and  use  network  topology  to  create  new  hybrid 
states  that  form  new  functions,  such  as  shielding  important  swarm  controlling 
agents. 

In  addition,  many  models  make  the  mathematically  simple  but  physically 
implausible  assumption  that  swarms  are  globally  coupled  (that  is,  each  agent 
is  influenced  by  the  motion  of  all  other  agents  in  the  swarm)  [30,  8,  7,  18,  41]. 
Global  coupling  is  easier  to  analyze  and  a  reasonable  assumption  in  cases  of 
high-bandwidth  communication,  with  a  sufficiently  small  number  of  agents.  In 
contrast,  we  are  interested  in  the  collective  motion  patterns  that  emerge  when 
global  communication  cannot  be  achieved.  New  behaviors  can  unexpectedly 
emerge  when  the  communication  structure  of  a  network  is  altered,  as  in  [46], 
where  the  stability  of  solutions  for  compromise  dynamics  over  an  Erdds-Renyi 
communication  network  is  considered.  However,  in  our  system,  we  show  robust¬ 
ness  of  emergent  motion  patterns  to  loss  of  communication  links  in  presence  of 
delayed  coupling  [37]. 

A  third  effect,  which  we  do  not  consider  here,  is  agent  heterogeneity.  Most 
existing  work  assumes  that  the  members  of  the  swarm  are  identical.  However, 
many  practical  applications  involve  swarms  that  are  composed  of  agents  with 
differing  dynamical  properties  from  the  onset,  or  that  become  different  over 
time  due  to  malfunction  or  aging.  Swarm  heterogeneity  leads  to  interesting  new 
collective  dynamics  such  as  spontaneous  segregation  of  the  various  populations 
within  the  swarm;  it  also  has  the  potential  to  erode  swarm  cohesion.  In  biol¬ 
ogy,  for  example,  it  has  been  shown  that  sorting  behavior  of  different  cell  types 
during  the  development  of  an  organism  can  be  achieved  simply  by  introducing 
heterogeneity  in  inter-cell  adhesion  properties  [34,  13].  In  robotic  systems,  al¬ 
lowing  for  heterogeneity  in  dynamical  behaviors  of  swarm  agents  gives  greater 
flexibility  in  system  design,  and  is  therefore  desirable  not  only  from  a  theoretical 
but  also  from  a  practical  point  of  view  [15]. 

In  this  preliminary  work  we  are  interested  in  swarms  that  are  interacting. 
That  is,  the  entire  swarm  is  split  into  two  parts  initially,  and  then  the  two 
components  approach  each  other  as  flocks.  We  then  are  interested  in  under¬ 
standing  the  results  based  on  physical  parameters,  such  as  attraction/repulsion 
length  scales,  delayed  communication,  and  coupling  strength.  The  model  for 
the  swarm  we  use  is  based  on  the  the  employed  in  [9],  which  describe  a  mathe¬ 
matically  swarm  model  using  the  Morse  potential.  Recently,  the  authors  in  [1] 
studied  the  effects  of  swarm  collisions  using  the  Morse  potential  as  a  function 
of  incident  angle  of  the  interacting  flocks.  However,  in  any  robotic  swarm  there 
must  be  some  delay  between  sensing  other  agents,  and  control  actuation.  We 
investigated  the  result  of  applying  delay  to  the  coupling  terms  of  a  swarm  model. 
As  a  result,  every  agent  behaves  as  though  the  other  agents  are  at  the  positions 
they  were  at  rtime  units  before.  We  found  that  the  flocking  state  is  unlikely 


2 


following  collisions  of  delay-coupled  swarms,  even  for  fairly  small  values  of  r. 
The  implication  is  that  it  should  be  possible  to  have  one  swarm  modify  the  in¬ 
tent  of  another  by  either  capturing  the  swarm,  or  redirecting  the  mean  direction 
of  the  flock. 

2  The  Basic  Swarm  Model 

The  swarm  model  we  used  for  this  preliminary  study  follows  the  model  in  [9]. 
It  is  a  system  of  N  agents  in  n-dimensional  space  with  position  vectors  Xi  £  Rra , 
acting  under  the  following  equation  of  motion: 


(1) 


3= 


where  a,/3,  A  are  constants,  U  :  M2rl  ->  K  is  a  potential  function  of  the  two 
agents’  position,  and  VXi  represents  taking  the  gradient  with  respect  to  X{. 
Many  potential  functions  can  be  used,  as  in  [27],  but  a  useful  choice  is  the 
scaled  Morse  potential: 


U(xi,  Xj)  =  C exp(—  \xi  —  Xj\/l)  —  exp(— jccj  —  Xj\). 


(2) 


In  particular,  the  parameter  l  can  be  used  to  set  a  ratio  of  repulsive  and  attrac¬ 
tive  length  scales. 

Following  [1],  we  fix  the  values  of  a,  /3,  C ,  and  l.  We  also  constrain  our 
swarm  to  the  plane.  We  choose  a  =  1,  /?  =  5,  C=^,i  =  |.  This  means  that, 
at  long  ranges,  V XiU(xi,  Xj)  will  be  attractive,  and  at  short  ranges  it  will  be 
repulsive  (see  Fig.  1).  Furthermore,  the  parameters  choice  places  the  swarm  in 
the  region  which  is  defined  as  ’’catastrophic”  [9],  meaning  that  increasing  the 
number  of  agents  decreases  the  space  between  agents,  so  that  the  total  size  of 
the  swarm  approaches  a  limit  in  space. 


(a)  Morse  potential  as  a  function 
of  distance. 


-0.0! 


0.11 


0.0! 


0.0C 

-0.C 


(b)  Morse  potential  of  a  variable 
point  on  a  plane  and  a  point  at 
the  origin. 


Figure  1:  The  Morse  potential  with  C  =  ^  and  l  =  | . 
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When  VXiU(xi,Xj)  =  0  for  all  i,j  <  N  with  i  ^  j,  the  dynamics  is  based 
solely  on  self-propulsion,  and  the  resulting  configuration  of  agents  is  called  a 
flock  [19].  That  is,  all  of  the  agents  when  started  in  parallel,  will  translate 
with  the  same  asymptotic  velocity  and  direction.  In  the  attraction-repulsion 
model,  a  flock  undergoes  translating  motion  at  a  constant  speed  y/a//3,  and 
can  withstand  sufficiently  small  perturbations  [6].  However,  due  to  translation 
symmetry,  there  will  always  be  one  eigenvalue  in  the  linearization  about  the 
flocking  state  equal  to  zero,  which  means  the  stability  is  neutral  at  best.  Hence 
small  perturbations  may  destabilize  the  flocking  dynamics  [25]. 

Since  we  are  interested  in  flock  collisions,  two  separate  flocks  are  initialized 
and  pointed  towards  each  other  with  some  incident  angle,  9.  We  are  interested 
in  the  behavior  after  collision,  and  how  it  varies  for  varying  A,  9,  and  r. 


Figure  2:  The  incident  angle  of  the  collision,  9 ,  which  we  vary  as  a  parameter. 


Flocking  is  not  the  only  behavior  that  swarms  display.  They  also  can  perform 
milling  behavior,  where  the  agents  rotate  around  a  stationary  center  of  mass. 
To  differentiate  between  the  two  behaviors,  we  calculate  the  polarization  of  the 
swarm  as  the  directed  sum  of  velocities.  Specifically,  for  a  system  of  N  agents 
with  position  vectors  x*,  the  polarization  P  of  the  flock  is  defined  as: 


P{x l,x2,  . .  •  ,XN) 


Two  limiting  behaviors  can  be  immediately  seen  to  be  reflected  in  the  polariza¬ 
tion.  If  the  velocities  are  coherent,  we  expect  a  value  of  P  ~  1.  If  the  velocities 
point  in  opposite  directions,  we  expect  P  «  0.  Therefore,  polarization  is  a  good 
measure  of  whether  the  system  displays  flocking  (P  «  1),  or  milling  (P  «  0). 
Additionally,  two  colliding  swarms  can  scatter,  in  which  they  do  not  form  a 
coherent  group  which  either  flocks  or  mills.  In  this  case,  we  expect  P  to  be 
neither  close  to  0  nor  close  to  1.  See  Fig.  3  for  examples  of  the  three  types  of 
swarm  interactions. 


3  Delay-Coupled  Swarms 

As  mentioned  above,  since  robotic  systems  possess  latency  in  both  communica¬ 
tion  and  control  actuation,  we  generalized  the  model  to  include  a  time  delay  r 
in  the  interaction  term,  creating  the  following  delay  differential  equation  from 
equation  (1): 
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(a)  Flocking. 


(b)  Milling. 


(c)  Scattering. 


Figure  3:  Flocking,  milling,  and  scattering  behavior  in  a  Morse  swarm. 


with  U  defined  in  Eq.  (2).  Since  this  is  a  delay  differential  equation,  it  requires  an 
initial  history  function  of  position  and  velocity  defined  over  [— r,  0] ,  for  which  we 
used  translating  motion  of  the  two  flocks  along  linear  trajectories.  We  simulated 
two  flocks  of  25  agents,  each  swarm  on  a  collision  path  with  the  other.  We  varied 
the  values  of  A,  the  coupling  strength,  and  0,  the  incident  angle  of  the  collision. 

As  a  benchmark,  we  first  simulated  the  results  by  performing  the  experi¬ 
ment  without  delay;i.e.,  r  =  0,  corresponding  to  a  system  with  instantaneous 
interactions.  We  simulated  400  collisions  and  recorded  the  polarization  after 
transients  were  removed,  about  100  time  units.  We  generated  the  plot  in  Fig.  4 
showing  how  polarization  can  be  used  to  quantify  the  resulting  dynamics  after 
the  swarms  collide. 

From  Fig.  4,  we  immediately  observe  several  large  scale  features  in  parameter 
space.  For  low  values  of  A,  scattering  is  likely  unless  the  swarm  motion  is  almost 
parallel  in  its  initial  state.  For  intermediate  values  of  A,  flocking  is  likely  to  be 
observed,  while  for  high  values  of  A,  milling  dominates  the  parameter  region. 
A  region  of  metastability  exists  between  the  flocking  and  milling  states.  The 
black  lines  are  used  to  guide  the  eye  to  show  the  boundaries  separating  the  final 
swarm  states. 

To  see  how  the  milling  comes  into  play  for  the  delay-coupled  swarm,  we  start 
the  flocks  approaching  each  other  at  zero  incidence  angle.  Figure  5  illustrates 
the  progression  from  the  initial  flocks  to  the  final  milling  swarm.  It  shows  the 
potential  of  how  one  might  use  a  defensive  swarm  to  stop  another  by  converting 
its  flocking  state,  which  is  a  traveling  center  of  mass,  to  a  milling  state  with  a 
center  of  mass  that  is  stationary. 

In  exploring  the  effects  of  delay  on  the  parameter  basins  of  swarm  final  states, 
we  computed  the  polarization  for  three  distinct  values  of  delay:  r  =  0,  0.1,  and 
0.2.  The  results  of  the  average  polarization  plots  using  the  above  procedure  are 
in  Fig.  6. 
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Figure  4:  Polarization  as  a  function  of  A  and  9.  Note  the  distinct  regions  of 
flocking,  milling,  scattering,  and  metastability. 


Figure  5:  Time  snapshots  of  swarm  capture  to  a  milling  state.  Delay  here  is 
r  =  0.1,  and  other  parameters  were  chosen  in  the  milling  regime  of  Fig. 


We  can  note  several  things  about  the  introduction  of  delay  in  panels  of 
Fig.  6.  Reference  from  the  r  =  0  case,  one  see  a  gradually  shrinking  of  the 
flocking  region,  with  an  increase  in  milling  parameters.  Even  at  the  relatively 
small  delay  of  r  =  0.2,  flocking  behavior  nearly  vanishes.  Instead,  scattering 
behavior  becomes  more  likely  for  low  A,  and  milling  behavior  becomes  more 
likely  for  high  A. 
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To  check  the  overall  dependence  of  swarm  final  state  on  both  coupling 
strength  and  delay,  we  fixed  the  incidence  angle  9  at  and  varied  r  finely, 
to  observe  polarization  as  a  function  of  r  and  A.  The  results  are  shown  in 
Fig.  7.  This  confirms  that  as  delay  increases  steadily,  milling  behavior  largely 
replaces  flocking  behavior. 


4  Conclusion 

The  preliminary  results  of  this  work  reveal  certain  features  about  how  interact¬ 
ing  swarms  upon  collision  modify  their  behavior.  One  major  finding  is  that  the 
existence  of  a  swarming  flocking  state  depends  strongly  on  the  delayed  interac¬ 
tion  between  agents.  If  delays  can  be  induced  into  the  communication  network 
of  a  flocking  swarm,  the  results  show  that  even  small  delay  disrupts  the  motion 
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Figure  7:  A  plot  of  polarization  as  a  function  of  A  and  r  for  fixed  6.  Note  that 
flocking  behavior  all  but  disappears  above  r  ss  0.1. 


of  the  center  of  mass,  thus  disabling  the  flocking  state. 

In  the  case  of  globally  coupled  interacting  swarms  which  are  equal  in  number, 
the  resulting  swarm  states  may  be  in  one  of  three  states:  Flocking,  Milling,  and 
Scattered.  However,  the  results  from  this  study  depend  heavily  on  the  initial 
states,  such  as  velocity  and  incidence  angle,  in  addition  to  delay  magnitude. 
Nonetheless,  the  final  states  of  colliding  swarms  may  result  in  modifying  the 
center  of  mass  velocity,  change  the  navigation  course  of  the  swarm  from  its 
original  path,  and  capture  another  swarm.  Th  potential  of  using  one  swarm 
to  modify  another  also  depends  on  how  it  is  sensing  other  agents,  which  could 
be  through  lidar,  vision,  or  from  a  third  party,  such  as  a  mother  ship  equipped 
with  radar. 

The  sensitivity  of  the  flocking  state  also  has  some  other  consequences  for 
modeling.  For  example,  it  shows  that  self-organizing  real  flocking  biological 
systems  which  may  be  modeled  by  swarms  may  have  more  complicated  under¬ 
lying  dynamics  and  adaptive  controls  than  are  not  included  in  the  deterministic 
equations  of  motion.  There  may  be  some  other  factors  not  modeled  here  which 
biological  flocks  may  use  to  compensate  for  actuation  delays,  noise,  and  com¬ 
munication  latency.  In  particular,  since  the  linear  stability  of  the  flocking  state 
is  neutral,  there  must  be  other  individual  agent  controls  to  make  the  flocking 
more  robust  to  perturbations. 

The  preliminary  results  here  also  show  that  globally  coupled  swarms  with  de¬ 
layed  communication  cannot  be  accurately  modeled  by  an  instantaneous  model. 
Similar  to  work  shown  in  [37],  delays  cause  new  states  to  emerge  that  are  not 
observed  in  systems  with  instantaneous  communication.  Moreover,  it  is  also 
possible  to  create  new  when  delay  is  included  in  the  coupling  terms  along  with 
changes  in  network  communication  topology  which  is  not  global,  as  shown  in 
[15].  In  the  case  of  a  given  Morse  potential  here,  simulations  accurately  mod¬ 
eling  delay-coupled  swarms  must  take  the  delay  into  account  to  get  accurate 
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results. 

Finally,  since  only  the  Morse  potential  was  used  here  to  model  interacting 
swarm  dynamics,  other  communication  and  coupling  schemes  need  to  be  studied 
to  see  what  kinds  of  general  statements  may  be  made,  and  which  conclusion 
pertain  only  to  specific  swarm  models.  The  end  results  of  such  research  should 
prove  to  open  new  areas  of  interacting  and  combating  swarms  in  the  future. 
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